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Standardized Tests as an Alternative to the Oral Interview 

by J. Sanf ord Dugan, Ph.D. 

This work is supported, in part, by a grant from the Department of 
Education under the Higher Education Act, Title VI. 

Introduction 

During the past seven or eight years, since the American Council on the 
Teaching of Foreign Languages undertook to promote a procedure and a 
scale for measuring speaking proficiency, the "Proficiency Movement" has 
won general recognition among foreign language trachers. A major 
contribution of ACTFL's effort has been to establish the Oral Proficiency 
Interview as the best known measure of speaking proficiency in the 
profession. Considerable attention has been paid in the literature of the 
past few years to studying this measure. There are few language teaching 
journals that have not publ'shed one or more articles on proficiency. 
Numerous textbooks have included a proficiency orientation. Many regions 
of the country have benefited from workshops designed to familiarize 
teachers with the Oral Proficiency Interview. Many individuals have 
earned credentials from ACTFL as certified testers in both commonly and 
less commonly taught langjages. The value of results from the Oral 
Proficiency Interview in terms of reliability seems fairly well 
established. In terms of validity there is controversy but not enough to 
prevent growing numbers of professionals from accepting it as a useful 
way to communicate about speaking proficiency. 

The one undisputed characteristic of the oral interview is, however, 
that it takes time. Normally, in the ACTFL model, 15 to 30 minutes are 
required to carry out the interview. If a single rater conducts the 
interview alone, the interview snould be recorded and rated by another 
qualified rater. Subsequent review by either or both raters, which may be 
necessary in borderline cases, adds even more time. Assuming an average 
time of 60 minutes per student, it can be seen that teachers dealing with 
upwards of 90 students a semester will not find this a practical procedure 
in situations where large numbers of students need to be tested in a short 
period, such as for placement at the start of a program or for final 
evaluation at the end of a program. However, if another procedure were 
shown to give similar results with more rapidly administered procedures, 
the latter might have some use. It is the purpose of this paper to present 
evidence of a correlation between the results of the oral proficiency 
o interview with two laboratory administered tests, one in the listening 
eric mode and one in the speaking mode. 
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Procedures 

In the winter term of 1984, with considerable cooperation and 
collaboration within the Department of Foreign Languages and Bilingual 
Studies at z medium-sized regional public university in the midwest, a 
group of sudents at various levels in French took the oral proficiency 
mterview and two laboratory tests, the MLA listening test and the MLA 
speaking test. It was made clear to the students that participation in the 
tests was a required part of the course but that the results of the tests 
would not count toward a grade. Following each test, students completed 
a survey composed of twelve questions asking their reactions to the test. 
The results of the tests and surveys are in the process of being analyzed 
and prepared in order to be submitted for publication. 

A. The Sample 

In the winter term 1984, the Department offered 15 French classes at 
four undergraduate levels and one graduate level. There were a total of 
23 i enrollments representing approximately 197 people. Since one person 
may register for more than one French course, there are more enrollments 
than people. There were 79 students at the 1 00 level; none of these were 
registered for classes at a higher level. There were 58 200-level 
students, one of whom was also registered in a course at a higher level. 
At the 300 level there was a total of 47 people, 7 of whom were enrolled 
in more than one course. At the 400, 500, and 600 levels, where two 
courses at different levels are often combined into one class, about 15 * 
people were enrolled in four classes. The percentage of participation at 
the different levels and overall is as follows: 100 level: 73%; 200 level: 
60%; 300 level: 67%; 400 level: 15% (approximately); 500, 600 (graduate) 
levels: 0%. The overall participation was 64%. 

B. The Instruments 

The Oral Proficiency Interview is a procedure and a rating scale that 
the American Council on the Teaching of Foreign Languages, in 
collaboration with the Educational Testing Service, has adapted from a 
model used for many years by U.S. government foreign language training 
agencies. ACTFL began offering workshops to train foreign language 
teachers in the academic sector to use a modified scale and procedure that 
are consistent with the levels reached by students of foreign languages in 
schools and colleges. Briefly, the interview takes place between a 
candidate and the tester. There is no fixed set of questions. Based on 
candidate responses to a few warm-up questions, the tester attempts to 
launch a conversation in which the candidate can show the highest level of 
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proficiency of which he or she is capable. The scale used has nine levels 
divided into four ranges; the ranges are called Novice, Intermediate, 
Advanced, and Superior. Once the tester is confident of having given ample 
opportunity for the candidate to speak and is confident of the rating to be 
given, the interview is brought to a close. Depending on the candidate*s 
proficiency, the interview may last from 15 minutes to as much as one 
half hour. Usually the interview is recorded, and, in order to gain an 
official rating from ACTFL, the interview is rated by another certified 
tester. In the present case, I carried out the interviews in the winter 
term of 1984, having participated in the ACTFL tester training workshop 
in the fall of 1983. I recorded the interviews for my own interest, but 
they were not reviewed by any others, and the ratings assigned are mine 
alone. 

The MLA tests of listening and speaking are a set of different tests at 
three different levels, L, M, and H, developed by the Modern Language 
Association in collaboration with the Educational Testing Service in the 
early 1960's. The listening test is on audio tape, lasts about 30 minutes, 
and can be administered to a large group in one sitting. The speaking test 
lasts about 15 minutes and requires that the candidate's responses be 
recorded on tape. It can be administered to a group in a language 
laboratory. These tests can be characterized as discrete-point item tests, 
that is, each question focuses on a very limited linguistic problem. The 
level L test was written for students near the end of one year of college 
French, level M for students near the end of two years, and level H for 
students near the end of a college major program preparing a career in 
teaching. In an ambitious norming study conducted by ETS in the 1960's, 
all levels of the tests were admnistered to students in selected schools 
around the country. The results were intercorrelated to create an overall 
converted score scale ranging from 100(?) to 300 (?). In the present 
study, students took both the listening test and the speaking test in the 
language laboratory. Students registered in French 121 and 122 took the 
listening and speaking tests at level L; students in French 222, 233, and 
234 took level M; and students in French 342, 344, and 362 and above took 
level H. For the listening test, all items were multiple-choice. The papers 
were scored and the scores were converted to the converted score scale. 
The speaking test responses were recorded on audio cassette, and each one 
was evaluated by myself following guidelines recommended in the MLA 
manual. The resulting scores were also converted to the convened score 
scale. 
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Students were given a survey consisting of a rating scale from one to 
five, with one representing strongly agree and 5 representing strongly 
disagree, and twelve statements. Students were instructed to rate their 
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reactions to all statements immediately following each of the tests. They 
were also asked to sign their copy of the survey. 

C. Analy sis 

Following the administration of the tests and gathering of results 
considerable time has been devoted to rating the speaking tests and to 
transfering data to a form that can be analyzed by computer. That work is 
still going on. Some preliminary results are available. 

Results 

A. Descriptive statistics 

Results of Tests by Semester Level 
Mean/Standard Deviation 



(N=Number of Subjects) 



Course 
Number 


Semester 
Level 


QR1 


MLA 
LSI 


MLA 

SPK 


FRN 121 


l 


2.3/1.0 
(N=32) 


151.8/10.1 
(N=36) 


169.4/12.1 
(N=33) 


FRN 122 


2 


2.9/1.3 
(N=15) 


162.6/9. 
(N=8) 


176.4/7.2 
(N=ll) 


FRN 222, 
234 


4 


4.2/1.3 
(N=52) 


167.0/1 1.9 
(N=54) 


172.9/6.0 
(N=52) 


FRN 362 


6 


5.7/1.2 
(N=19) 


175.8/14.9 
(N=23) 


186.2/5.2 
(N=23) 


FRN 


6-9 


5.7/1.3 
(N=10) 


184.9/17.6 
(N=15) 


188.1/8.0 
(N=16) 
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As can be seen, there is a clear progression in the test results according 
to the semester level of enrollment. This is true for all of che tests. Just 
how close this correlation is can be. seen in a correlation coefficient, as 
follows: 

B. Correlations 

Correlation tables for the Oral Proficiency Interview (OPI), the MLA 
Speaking Test (SPK), and the MLA Listening Test (LIS); N is the number of 
cases available and p is the sionificance level. 

FRN 121 (Pearson r) 

OPI 

OPI 

SPK 0.72 
(N=28) 

LIS 0.73 
(N=3 1 ) 



SPK 



LIS 



0.80 
(N=33) 



p<0.00l 



FRN 122 (Pearson r) 

OPI SPK LIS 

OPI 

SPK 0.49 
(N=25) 

LIS 0.64 0.65 p<0.0l 

(N=27) (N=30) 




FRN 222 (Pearson r) 

OPI SPK LIS 

OPI 

SPK 0.64 
(19) 

LIS 0.53 0.47 p<0.1 

(20) (20) 7 
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FRN 734 (Pearson r) 

OPI SPK LIS 

OPI 

SPK 0.60 
(31) 

LIS 0.69 0.74 p<0.001 

(32) (32) 



FRN 342, 344, 36? (443/620) (Pearson r) 

OPI SPK LIS 

OPI 

SPK 0.75 
(N=28) 

LIS 0.68 0.74 p<0.001 

(N=28) (N=36) 



All Levels (Spearman rho) 

OPI SPK LIS 

OPI 

SPK 0.61 

(N=I33) 

LIS 0.74 0.71 p<0.01 

(N=139) (N=153) 

The N value, that is, the number of people whose scores are used in the 
correlation, is different within a given class, because, when one person 
did not take both tests being correlated, that person's score is not 
$ included. In each table there are also correlation values and a confidence 

ERJC level. In mosUtables the highest correlation is between the MLA Listening 
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and the MLA Speaking tests. This is expected, since the two tests were 
developed together by the Educational Testing Service. In four of six 
tables the lowest correlation is between the MLA Speaking Test and the 
Oral Proficiency Interview. This is surprising, but may b explained by the 
fact that scoring the MLA Speaking Test is complicated, and maintaining 
reliability is very difficult. The most interesting correlation is between 
the Oral Proficiency Interview and the MLA Listening Test. In only one 
class did the value fall below 0.60. That class, FRN 222, is devoted mostly 
to reading French; there is little emphasis on listening/speaking abilities, 
which are more important in ail the other classes. In the correlation of 
scores at all levels the MLA Listening with Oral Proficiency Interview 
value shows a moderately strong correlation. The confidence values are 
satisfactory, except in the case of FRN 222, where p=0.1. This means that 
the result may not be significant statistically and may not merit being 
considered. 

In a related study, thirteen students from a 300-level class were 
candidates for the Chamber of Commerce and Industry of Paris Certif icat 
de francais des affaires. To obtain the Certif icat, students must 
successfully pass an examination that takes a full day and includes four 
written parts and two oral parts. As this exam was administered at E.M.U., 
the results of the oral part of the test were available. When these results 
are correlated with the results of the Oral Proficiency interview, a 
Pearson r of 0.67 with a p value of <.01 is observed. 



Discussion 



There seems to be more than a hint that the MLA listening test and the 
MLA speaking test have a moderate correlation with the ACTFL/ETS Oral 
Proficiency Interview. This suggests that, in some circumstances, one of 
these tests might be used to replace the other. For example, when there is 
a need to screen large numbers of candidates in a short period of time, as 
at the begnning of a program of instruction, the listening test, which can 
be administered to considerable numbers in one sitting and scored rapidly, 
even by machine, might serve as a basis for preliminary judgment of 
individuals' proficiency in speaking. Another possible use of the apparently 
more efficient procedure would be to help candidates who are preparing 
for an oral interview that may be a requirement for successful completion 
of a program of instruction. 

While the MLA Listening Test is morfcefficient, there is a problem that 
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must be addressed if it is used for placing students in a program. This 
test is really three tests at different levels. Although the scores are 
convertible to a converted score scale, some idea of the student's prior 
background in the language is needed in order to chose the test whose 
score will show most effectively the student's true level. Such 
information is not always available or interpretable, as in the case of 
someone who has not followed a regular program of study. This is less of 
a problem when the MLA Listening test is used within a given program 
where the student's progress is already known. 

Some ways that this study might be improved if it were to be run again 
would be to have more than one rater for the oral proficiency interview 
and to have independent raters, that is, one who are not directly involved 
in teaching the students to be tested. Another improvement would be to 
base the identifying of the sample group on a random selection procedure. 

It will be interesting to see whether the results of the student attitude 
survey will show a preference for any one of the tests or whether students 
seem to view them as equally valid for testing their proficiency in the 
language. 

Conclusion 

The presenting of information on a moderate correlation between the 
time-consuming face-to-face Oral Proficiency Interview and the more 
efficiently administered and scored MLA Listening test is perhaps not 
surprising, given the observation that language learning has at least a 
general factor. That is to say, the learning of different skills, such as 
speaking and listening, involves, to some extent, learning a combined or 
"general" skill in the language. Given a normal sample of language 
students over a wide range of levers, those individuals who are more 
proficient in listening will most probably be more proficient in speaking. 
Furthermore, greater efficiency with similar results does not make one 
test better for use in all situations. The oral interview has a flexibility 
allowing personal interchange between tester and candidate that is 
impossible in the language lab. This opportunity for communication, when 
time constraints are not as much of a factor, is a distinct advantage 
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ALUMNI QUESTIONNAIRE 

specify other direc.w..~, 
the letter(s) of the most appropriate responses(s 



For items which do not specify other directions, please circle 



PERSONAL AND EDUCATIONAL DATA 

1. What is your citizenship? 

a. U.S. 

b. Other 



2* What degree/certification did you 
complete at EMU? 

a* Bachelor 1 s degree 

b. Master's degree 

c. Elementary certification 

d. Secondary certification 

e. K-12 certification 

3. What was your major area of study? 
(If Language and International Trade, 
circle language area as well.) 

a. English as a Second Language 

b. French 

c. German 

d. Spanish 

e. Bilh.jual Education 

f. Language and International Trade 

g. TESOL 

k. When did you complete your program? 
Month 19 

5. Have you taken any certification or 
competency examinations related to your 
program of study? 

a. No 

b. Yes, and I passed. 

c. Yes, and I did not pass. 

d. Yes, but I do not yet know the result, 

6. If you answered "Yes 11 to #5, which 
examination di.d you take? 



8. If you answered "Yes 15 to #7, what 
further degree or certificate have 
you earned or do you plan to earn? 



9. If you answered #8, 

A. What was/is your main area of study? 



B. What was/is your reason for further- 
ing your studies? 



10. Did you complete an Internship, 
cooperative education, or student 
teaching assignment at EMU? 

a. Yes, in the U.S. 

b. Yes, overseas in 

c. No. 
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If you answered "Yes" to #13: 
A* Name of company or school: 



B. From 



(Month/YrT 



to 



(Month/Yr) 



12. 



How would you rate EMU assistance 

in locating your off-campus assignment? 



a. Excel lent 

b. Good 



c. Fair 

d. Poor 



7. Since completing your program at EMU, 
have you continued, or do you plan to 
continue, your education? 

a. Yes, I have completed further study. 

b. Yes, I am currently enrolled in a program. 

c. Yes, I plan to continue my education. 

d. No, I have no such plans. 



13. 



How would you rate your off-campus 
assignment? 

a. Excellent d. Poor 

b. Good e. Waste of Time 

c. Fair 
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\k. Objectives , Please Indicate whether your prog-am at EMU helped you meet the 
following objectives by checking the most appropriate response. 

Academic Objectives Yes No N/A 

a. Increase knowledge and understanding of your field of study 

b. Obtain a degree or certificate 

c. Apply theoretical learning 

d. Identify important professional organizations and sources of 

information 

Career Education and Preparation Yes No N/A 

e. Discover career interests 

f. Develop long-term career goals 

g. Prepare for a career change 

h. Obtain skills for your career 

i. Increase chances for promotion 

15. Which of the above objectives (or others you have/had) were the most important 
during your studies and which do you think are the most important now? 

During studies Now 
Most important 

Second 

Third 

EMPLOYMENT INFORMATION 



16. Are you currently employed? 

a. Yes, full-time 

b. Yes, part-time 

c. Ko 



19. If not the same, what is your current 

job, or if unemployed, your most recent job? 

a. Area of employment (refer to #17) 



If yes, how long have you been 

In this position? 

b. Description of position (i.e. duties) 

17. In what area was the first job 

you had after completion of your 

EMU program? 

a. Government e. Junior college 

b. Business/Industry f. University 

c. Public school g. Vocational school 

o\ Private school h. Adult education 

20. What is your current annual salary? 

18. Describe your position (accountant, ( F °r part-time jobs, estimate full-time 
elementary teacher), listing the three equivalent.) 

most Important duties: a# Less than $20, 000 c. $30,000— $39,000 

b. $20,0*0--$29,000 d. $^0,000 or wore 
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21. Was knowing a foreign language a signi- 25. If you answered positively to #2'4, 
flcant factor In your finding employment? which language? 

a. Yes b. No 

26. If you use a language in your job, 

22. To what extent Is your current (or most for what ^urposefs)? 
recent) job related to your major area 
of study? 

a. Directly related c. Not related 

b. Somewhat related at all 

23. If you answered "Not at all" to #22, 
why not? 

a. I never looked for work related to 
my area of study. 

b. I looked for a job in my area, but 
could not find one. 

24. To what extent do the functions of 
your present (or most recent) job 
Involve using your foreign language? 

a. More than k hours a day 

b. 1 to k hours a day 

c. Once a week 

d. Once a month 

e. None 

SUMMARY QUESTIONS 

28. How would you rate your overall language and culture preparation at EMU for 
your current (most recent) jbb? 

a. Excellent b. Good c. Adequate d. Inadequate 

29. What were the strengths of the EMU program in preparing you for your current 
work and/or studies? 



30. What were the weaknesses of the EMU program in preparing you for your current 
work and/or studies? Are there essential courses or area* of studies that need 
to be developed or added to the language/trade program? if so, what are they? 



. All personal information supplied in answer to the above questions will remain 
strictly confidential. Alumni of the Language and Internatio nal Trade program 
please go on to page k for more specific questions regarding your area. 



a. Reading d. Listening 

b. Writing e. Translating 

c. Speaking f. Interpreting 

27. How did you find your current (most 
recent) job? 

a. EMU Placement Office 

b. EMU professor contact 

c. Professional organization 

d. Employment agency 

e. Classified ad 

f. Direct application 

g. Friends/relatives 

h. Through co-op/internship 

i. Other 
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LANGUAGE AND INTERNATIONAL TRADE QUESTIONNAIRE 



The following ten items are specifically to be 
are in Language and International Trade. 

1. To what extent is your employer involved 
In international concerns? 

a. 76 - 100% 

b. 51 - 75% 

c. 26 - 50% 

d. 1 - 25% 

e. Not at all 

!f you answered one from a-d, with 
which countries? 



2. What is the primary activity of your firm? 

a. Manufacturing (non-auto) 

b. Auto manufacturing 

c. Retai 1/ Wholesale 

d. Finance/ Banking 

e. Transportation 

f. Other 



3. What are your firm's annual sales? 
(in millions of dollars) 

a. Under 10 

b. 10 - 25 

c. 25 - 50 

d. 50 - 100 

e. 100 - 250 

f. 250 - 500 

g. 500 - 1 billion 

h. Over 1 bi 1 1 ion 
i • I don 1 1 know 

How many employees does your firm 
have in the U.S.? 

a. Under 100 

b. 100 - **99 

c. 500 - 999 

d. 1000 and over 

e. I don't know. 



answered by those whose degrees 



5. Does your .firm have overseas subsi- 
diaries? 

Yes No 



6. How many employees does your firm 
have outside the U.S.? 

a. Under 100 

b. 100 - ^99 

c. 500 - 999 

d. 1000 and over 

e. I don't know. 

7. What is the primary activity of your 
firm in international business? 

a. Export 

b. Import 

c. Import-Export 

d. Finance 

e. Freight Forwarding 
d. Other 



8. To what extent are the duties of 
your job related to int'l concerns? 

a. 76 - 100% 

b. 51 - 75% 

c. 26 - 50% 

d. 1 - 25% 

e. Not at all 

9. In which business area did you 
concentrate? 

a. Accounting d. Marketing 

b. Finance e. Other 

c. Management 



10. To what extent were your business 
studies a factor In your finding 
a job? 

a. Very Important c. Slightly Important 

b. Important d. Unimportant 
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